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Dear  Senator  Roth: 


At  your  request,  we  examined  the  ALR-67(V)3  radar  warning  receiver 
(rwe)  program.  The  ALR-67(y)3  is  intended  to  help  protect  Navy  aircraft 
and  the  program  is  expected  to  cost  about  $1  billion.  We  are  issuing  this 
report  to  bring  to  your  attention  certain  aspects  of  the  Navy’s  acquisition 
plans  that  we  believe  will  unnecessarily  increase  the  Department  of 
Defense’s  (dod)  risk  on  the  program. 


Background 


RWRs  are  electronic  warfare  devices  that  help  protect  aircraft  against 
radar-controlled  weapons,  rwrs  sense  the  signals  from  hostile  radars, 
provide  an  audio  warning  to  the  pilot,  and  display  the  warning  information 
on  a  video  screen  in  the  cockpit.  The  display  identifies  the  threats, 
provides  their  location  or  relative  bearings,  and  ranks  the  threats  in  order 
of  danger  to  the  aircraft.  Based  on  the  warning,  the  pilot  chooses  fi*om 
various  options  for  defeating  the  threat,  such  as  employing  electronic 
jamming  to  interfere  with  the  radar. 

The  ALR-67(V)3  is  intended  to  be  the  Navy’s  next  generation  rwr  for  use 
on  its  future  F/A-18  E/F  aircraft.  The  ALR-67(V)3  is  also  planned  for  use  on 
some  current  F/A-18  C/D  aircraft,  and  with  minor  antenna  modifications, 
can  also  be  retrofitted  as  the  ALR-67(V)4  into  the  F-14  and  AV-8B  aircraft. 

'The  Navy  expects  the  ALR-67(V)3  to  significantly  outperform  the  current 
RWR,  the  ALR-67(V)2.  The  AliR-67(V)3  is  expected  to  be  capable  of 
detecting  threats  at  a  greater  range  and  identifying  more  types  of  threat 
radars  than  its  predecessor. 


Results  in  Brief 


Despite  numerous  performance  problems  that  surfaced  in  developmental 
testing  of  the  ALR-67(V)3,  the  adverse  consequences  from  the  premature 
procurement  of  the  original  ALR-67  and  the  ALR-67(y)2,  and  having 
already  produced  sufficient  test  articles  for  all  operational  testing,  the 
Navy  plans  to  conunit  to  low-rate  production  of  the  ALR-67(V)3  before 
determining  through  completion  of  operational  testing  that  the  system  is 
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operationally  effective  and  operationally  suitable.^  As  a  result,  the  Navy 
risks  procuring  a  deficient  system  that  may  require  expensive 
modifications  and  retrofit  to  achieve  adequate  performance. 


Adverse  Impact  of 
Premature  Production 
ofALR-67RWRs 


Despite  the  importance  of  operational  testing^  as  a  management  control  to 
ensure  adequate  system  performance,  dod  started  production  of  the 
original  ALR-67  and  its  successor,  the  ALR-67(V)2,  before  they  were 
operationally  tested.  The  adverse  consequences  included  deploying  an 
unsatisfactory  system  to  the  operational  forces  and  placing  newly 
produced  systems  in  storage  rather  than  having  them  enhance  the  Navy’s 
combat  posture. 


The  Navy  began  producing  the  original  ALR-67  in  the  early  1980s  before 
proving  it  was  operationally  effective.  Subsequently,  the  system  performed 
so  poorly  during  operational  testing  that  the  testing  had  to  be  cvirtailed. 
However,  the  Navy  continued  production  while  the  system’s  performance 
problems  remained  unresolved. 

As  a  result,  the  Navy  installed  the  unsatisfactory  system  in  operational 
aircraft  for  use  by  the  combat  forces.  Subsequently,  some  Navy  pilots 
during  Operation  Desert  Storm  distrusted  the  system  to  the  extent  that 
they  stopped  using  it  to  detect  threat  radars  and  relied  instead  on  other 
means  that  dod  considers  classified. 


To  solve  the  ALR-67’s  problems,  the  Navy  acquired  the  ALR-67(V)2.  Once 
again,  however,  the  Navy  started  production  before  correcting 
performance  deficiencies.  After  operational  testing  showed  that  the 
ALR-67(V)2  did  not  meet  the  Navy  requirement,  some  230  production 
units,  costing  $75  million,  were  placed  in  storage  pending  modifications  to 
achieve  satisfactory  performance. 


^Operational  effectiveness  refers  to  the  ability  of  a  system  to  accomplish  its  mission  in  the  planned 
operational  environment.  Suitability  is  the  degree  to  which  a  system  can  be  placed  satisfactorily  in 
field  use  considering  such  factors  as  reliability  and  maintainability. 

^Operational  testing  is  DOD’s  primary  means  of  evaluating  weapon  system  performance.  It  is  also  a  key 
internal  control  to  ensure  that  decisioiunakers  have  objective  information  available  on  a  weapon 
system’s  performance  to  minimize  risks  of  procuring  costly  and  ineffective  systems. 
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Developmental  Tests 
Reveal  Problems  With 
ALR-67(V)3  Prototype 


The  Navy  is  developing  the  ALR-67(V)3  to  overcome  the  problems  with  the 
previous  versions  of  the  ALR-67.  However,  developmental  testing  of 
ALR-67(V)3  prototypes  conducted  so  far  has  disclosed  numerous 
deficiencies  with  its  performance.  According  to  the  System  Anomaly 
Reports^  the  Navy  provided  to  us,  some  of  the  deficiencies  include 


•  misidentification  of  threat  radars  or  identification  of  friendly  radar  as  a 
threat, 

•  warnings  of  multiple  threats  when  only  one  is  present  or  vice-versa,  or 

•  failure  to  detect  radars  tracking  the  aircraft. 

Moreover,  developmental  tests  show  that  it  has  not  been  proven 
compatible  with  the  F/A-18  aircraft’s  radar.  For  example,  normal  operation 
of  the  F/A-18  radar  can  cause  the  ALR-67(V)3  to  warn  of  nonexistent 
threats. 


The  System  Anomaly  Reports  indicate  that  some  of  the  reported 
deficiencies  were  identified  as  early  as  mid-1993,  and  the  contractor’s 
recommended  action  for  many  of  the  reported  deficiencies  was  not  to 
proceed  to  operational  evaluation  until  corrected.  Without  operational 
evaluation,  however,  the  Navy  cannot  have  complete  assmrance  that  any 
planned  fixes  for  those  deficiencies  will  work. 

In  addition  to  the  developmental  tests  of  effectiveness  conducted  earlier, 
the  Navy  conducted  an  operational  assessment  of  a  developmental  model 
of  the  ALR-67(V)3  in  September  1994.  However,  because  the  system  was 
not  production  representative,  that  is,  it  was  not  of  the  final  design 
planned  to  be  produced,  the  Navy  deferred  testing  of  operational 
suitability.  Thus,  the  Navy  has  no  assurance  yet  that  suitability 
considerations,  such  as  reliability  and  maintainability,  will  be  adequate. 


Legislative 

Requirement  Imposed 
on  ALR-67(V)3 
Program 


Because  of  concerns  that  the  Navy  was  proceeding  with  a  high-risk 
acquisition  plan  for  the  ALR-67(V)3,  the  Congress  included,  among  other 
requirements  in  the  fiscal  year  1995  Department  of  Defense  Appropriation 
Act,  a  requirement  that  dod  certify,  based  on  the  results  of  joint 
developmental  tests  and  operational  test  flights,  that  the  ALR-67(V)3  was 
potentially  operationally  effective  and  potentially  operationally  suitable 
before  low-rate  production.  'This  requirement  was  applicable  only  dming 
fiscal  year  1995.  In  its  comments  on  a  draft  of  this  report,  however,  the 


^System  Anomaly  Reports  are  official  records  created  to  document  instances  when  a  system  fails  to 
perform  as  expected. 
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Navy  states  that  it  will  adhere  to  this  and  other  fiscal  year  1996 
requirements  pertaining  to  the  ALR-67(V)3. 

The  Navy  has  defined  an  operational  test  program  for  the  ALR-67(V)3 
calling  for  a  first  phase  of  joint  developmental  tests  with 
1  month  of  operational  test  flights,  which  it  calls  OT-EA.  OT-IIA  will  occur 
primarily  in  the  Navy’s  indoor  chamber  facility  at  Patuxent  River, 

Maryland,  and  is  planned  to  be  completed  in  August  1996.  This  is  to  be 
followed  by  a  4-month  phase  of  operational  flight  testing  called  OT-IIB  that 
is  planned  for  the  Navy’s  open  air  range  at  China  Lake,  California.  The 
Navy’s  program  schedule  indicates  that  OT-EB  is  scheduled  to  be 
completed  during  the  spring  of  1997.  According  to  the  program  office, 
potential  fixes  for  the  developmental  problems  described  above  will  be 
tested  and  confirmed  using  20  production  representative  assets  procured 
specifically  for  the  operational  tests. 


ALR-67(V)3  Schedule 
Contains  Unnecessaiy 
Risk 


Notwithstanding  the  unresolved  deficiencies  in  the  ALR-67(V)3’s 
effectiveness,  the  unknovm  nature  of  its  suitability,  and  the  Navy’s 
experiences  with  earlier  versions  of  the  ALR-67,  the  Navy  plans  to  commit 
to  the  ALR-67(V)3’s  low-rate  initial  production  before  completing  the 
second  phase  of  operational  testing  (OT-IIB)  in  the  spring  of  1997. 
According  to  the  program  schedule,  the  Navy  plans  to  award  a  low-rate 
production  contract  in  the  first  quarter  of  fiscal  year  1997  for  the 
manufacture  of  ALR-67(V)3  systems  for  deployment  on  F/A-18  aircraft 
after  the  first  phase  (OT-IIA)  of  joint  developmental  and  operational 
testing.  However,  while  OT-DA  may  demonstrate  potentialities,  the  Navy 
will  not  be  able  to  conclude  from  this  testing  phase  alone  that  the  system 
is  operationally  effective  and  suitable;  such  a  conclusion  must  be  based  on 
a  large  number  of  flight  hours  in  a  realistic  environment  that  only  OT-IIB 
can  provide.  Furthermore,  because  the  Navy  intends  to  conduct  both 
operational  test  phases  using  ALR-67(V)3  production  representative 
systems  previously  procured  as  test  assets,  low-rate  production  of 
additional  new  systems  is  not  needed  to  complete  operational  testing. 


While  the  Navy’s  stated  intent  to  adhere  to  an  earlier  requirement  that  the 
ALR-67(V)3  be  potentially  effective  and  suitable  may  provide  some 
assurance  of  future  success,  we  do  not  believe  that  this  goes  far  enough. 
Past  failures  with  earlier  versions,  unresolved  deficiencies  with  the 
ALR-67(V)3,  and  the  availability  of  test  assets  for  all  operational  tests, 
leads  us  to  conclude  completion  of  OT-IIB  rather  than  an  early 
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commitment  to  low-rate  initial  production  after  OT-IIA  would  be  a  far 
more  prudent  acquisition  strategy. 


Recommendation 


To  minimize  the  risk  of  procuring  another  deficient  rwr,  and  because  the 
Navy  does  not  need  additional  ALR-67(y)3  units  to  complete  operational 
testing,  we  recommend  that  the  Secretary  of  Defense  require  that  the 
ALR-67(V)3  complete  both  phases  of  operational  testing  (OT-IIA  and 
OT-IIB)  to  determine  its  effectiveness  and  suitability,  and  that  the 
deficiencies  identified  during  developmental  testing  have  been  resolved 
before  committing  to  low-rate  production. 


Matter  for 

Congressional 

Consideration 


Because  the  Navy  has  not  modified  what  we  believe  is  a  high-risk 
acquisition  strategy,  the  Congress  may  wish  to  restrict  dod  fl-om  spending 
any  funds  on  ALR-67(V)3  production  imtil  both  phases  of  operational 
testing  (OT-BA  and  OT-HB)  are  complete  and  the  system  is  certified 
effective  and  suitable  by  dod. 


Agency  Comments 


DOD  generally  concurred  with  this  report,  but  indicated  it  did  not  agree 
with  our  draft  report  recommendation  that  the  Secretary  of  Defense 
require  the  Navy  to  do  sufficient  testing  before  production  to  ensure  that 
developmental  deficiencies  have  been  resolved,  dod  maintained  that 
because  of  the  congressional  reporting  requirements,  it  saw  no  need  to 
place  any  additional  requirements  on  the  Navy.  However,  when  the 
requirements  were  enacted,  the  deficiencies  identified  during 
developmental  testing  were  not  generally  known.  As  we  stated  in  this 
report,  the  Navy’s  intent  to  adhere  to  the  earlier  requirements  may  provide 
some  assurance  of  future  success,  but  in  light  of  the  unresolved 
deficiencies,  the  Navy’s  continued  adherence  to  its  acquisition  strategy  is 
not  reassiiiing.  We  have,  therefore,  modified  our  report  and 
recommendation  to  address  the  congressional  reporting  requirements  and 
restate  our  concern  that  the  Navy  continues  to  pursue  a  high-risk 
acquisition  strategy  by  not  completing  both  phases  of  operational  testing 
before  low-rate  production  and  by  planning  to  acquire  additional  systems 
not  needed  for  testing,  dod’s  comments  are  presented  in  their  entirety  in 
appendix  I  along  with  our  evaluation  of  them. 


Scope  and 
Methodology 


We  performed  our  work  at  the  Naval  Air  Systems  Command,  Washington, 
D.C.;  the  Offices  of  the  Director  of  Electronic  Combat  and  Director  of 
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Operational  Test  and  Evaluation  (dot&e)  and  the  Office  of  the  Secretary  of 
Defense  (osd),  Washington,  D.C.  In  evaluating  ALR-67(Y)3  performance, 
we  reviewed  developmental  test  results,  including  System  Anomaly 
Reports  and  operational  assessment  results.  We  also  discussed  the  test 
results  and  potential  performance  issues  with  dot&e,  osd,  and  Navy 
officials  responsible  for  oversight  of  electronic  warfare  systems’ 
acquisition. 

Our  review  was  performed  from  January  1996  through  April  1996  in 
accordance  with  generally  accepted  government  auditing  standards. 


We  are  sending  copies  of  this  report  to  other  appropriate  congressional 
committees;  the  Secretaries  of  Defense  and  the  Navy;  and  the  Director, 
Office  of  Management  and  Budget.  We  will  make  copies  available  to  others 
upon  request. 

Please  contact  me  at  (202)  5124841  if  you  or  your  staff  have  any  questions 
concerning  this  report.  Major  contributors  to  this  report  were  Jackie  B. 
Guin,  Charles  A.  Ward,  and  A.  Delores  Cohen. 

Sincerely  yours. 


Louis  J.  Rodrigues 

Director,  Defense  Acquisition  Issues 
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Comments  From  the  Department  of  Defense 


Note:  GAO  cominents 
supplementing  those  in  the 
report  text  appear  at  the 
end  of  this  appendix. 


See  comment  1 . 


See  comment  2. 


ACQUISITION  AND 
TECHNOLOGY 


OFFICE  OF  THE  UNDER  SECRETARY  OF  DEFENSE 

3000  DEFENSE  PENTAGON 
WASHINGTON  DC  20301*3000 


MAR  0  7  1996 


Mr.  Louis  J.  Rodrigues 
Director,  Defense  Acquisition  Issues 
National  Security  and  International 
Affairs  Division 
General  Accounting  Office 
Washington,  D.C.  20548 


Dear  Mr .  Rodrigues : 

Enclosed  is  the  Department  of  Defense  (DoD)  response 
to  the  General  Accounting  Office  (GAO)  draft  report, 
"ELECTRONIC  WARFARE:  Navy'S  New  Radar  Warning  Receiver  Needs 
More  Testing,"  dated  February  5,  1996  (GAO  Code  707104),  OSD 
Case  1089. 

The  DoD  generally  concurs  with  the  report  but  disagrees 
with  the  GAO  assertion  that  the  Navy  intends  to  commit  to  the 
ALR-67(V)3  production  before  completing  operational  testing. 
The  ALR-67(V)3  will  not  go  into  low  rate  initial  production 
(LRIP)  until  the  Director,  Operational  Test  and  Evaluation 
reports  to  the  Congress  on  the  adequacy  of  operational  testing 
and  the  Navy  reports  to  the  Congress  on  the  objectives  and 
results  of  the  developmental  flight  testing. 


The  DoD  believes  the  recommendation  that  the  Secretary  of 
Defense  require  the  ALR-67 (V) 3  undergo  sufficient  operational 
testing  to  demonstrate  that  the  deficiencies  identified  during 
developmental  testing  have  been  resolved  before  committing  to 
LRIP  is  superfluous.  The  Navy's  strategy  for  operational 
testing  of  the  ALR-67 (V) 3  prior  to  initiating  LRIP  is  consistent 
with  good  acquisition  planning,  and  it  also  accommodates  the 
above  congressional  reporting  requirements  prior  to  the  LRIP 
decision.  The  Department  sees  no  need  to  place  any  additional 
requirements  on  the  Navy  regarding  the  ALR-67  (V)  3  prior  to  the 
LRIP  decision. 


Detailed  comments  regarding  the  report  and  the 
recommendation  are  enclosed.  The  Department  appreciates  the 
opportunity  to  comment  on  the  draft  report. 

Sincerely, 


George  R.  Schneiter 
Director 

Strategic  and  Tactical  Systems 


Enclosures 


Page  8 


GAO/NSIAD-96-68  Electronic  Warfare 


Appendix  I 

Comments  From  the  Department  of  Defense 


See  comment  1 . 


See  comment  3. 


See  comment  4. 


GAO  DRAFT  REPORT  -  DATED  Fobruary  5,  1996 
(GAO  CODE  707104)  OSD  CASE  1089 

"ELECTRONIC  WARFARE:  NAVY'S  NEW  RADAR  WARNING  RECEIVER 
NEEDS  MORE  TESTING" 

DEPARTMENT  OF  DEFENSE  COMMENTS 


***** 

RECOMMENDATION 

RECOMBtENDATION :  The  GAO  recommended  that  the  Secretary  of 

Defense  require  that  the  ALR“67  (V)  3  undergo  sufficient 
operational  testing  to  demonstrate  that  the  deficiencies 
identified  during  developmental  testing  have  been  resolved 
before  committing  to  low  rate  initial  production  (LRIP) . 

(p,  8 /GAO  Draft  Report) 

POD  RESPONSE:  Nonconcur.  The  test  program  for  the 
ALR’-67<V)3  has  been  structured  to  provide  the  necessary 
information  to  support  an  LRIP  decision.  Furthermore, 
congressional  language  in  House  Report  4650,  referenced  by 
Senate  Report  103-321  in  the  1995  Department  of  Defense 
Appropriation  Bill,  requires  both  the  Director,  Operational 
Test  and  Evaluation  (DOT&E)  and  the  Navy  to  submit  reports 
to  the  Congress  prior  to  the  ALR-67  (V)  3  entering  LRIP.  The 
Test  and  Evaluation  Master  Plan  (TEMP)  and  test  concept 
approved  in  December  1994  contain  a  well  structured  test 
program  which  was  developed  with  participation /oversight 
from  DOT&E  through  the  Navy's  Test  Planning  Working  Group. 
The  TEMP  defines  a  test  program  calling  for  an  early 
operational  test  (OT-IIA)  made  up  of  two  phases.  The  first 
phase  is  a  combined  development  and  operational  test  using 
installed  equipment  testing  conducted  primarily  in  the 
chamber  facility  at  Patuxent  River,  Maryland-  The  second 
phase  is  a  flight  test  conducted  primarily  at  the  Electronic 
Combat  Range  at  China  Lake,  California.  The  LRIP  decision 
will  be  based  on  a  total  program  review  including  the 
results  of  the  OT-IIA  and  the  subsequent  submission  of  the 
required  reports  by  DOT&E  and  the  Navy  to  the  Congress. 

Specific  language  of  House  Report  4650,  referenced  by  Senate 
Report  103-321  in  the  1995  Department  of  Defense 
Appropriation  Bill,  requires  that  the  congressional  defense 
committees  must  be  in  receipt  of  three  special  reports  at 
least  30  days  prior  to  the  Navy  obligation  of  the  FY  1995 
funding  for  LRIP  of  the  ALR-67{V)3.  DOT&E  is  required  to 
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report  the  adequacy  of  operational  testing  and  the  potential 
operational  effectiveness  and  suitability  of  ALR-67(V)3;  the 
Commander  of  the  Navy's  Operational  Test  and  Evaluation 
Force  is  required  to  report  on  the  readiness  of  the 
ALR-67(V)3  for  LRIP;  and  the  Assistant  Secretary  of  the  Navy 
for  Research,  Development,  and  Acquisition  is  required  to 
report  on  the  objectives  and  results  of  developmental  flight 
testing  and  the  technical  evaluation  along  with  a 
certification  of  readiness  to  enter  LRIP. 
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The  following  are  gao’s  comments  on  the  Department  of  Defense’s  (dod) 
letter  dated  March  7, 1996. 


GAO  Comments 


1.  According  to  the  ALR-67(V)3  program  schedule  provided  to  us  by  the 
program  office,  the  Navy  plans  to  begin  low-rate  production  of  the  system 
after  the  first  of  two  phases  of  testing  (OT-IIA).  dod  considers  OT-ILA  to  be 
joint  developmental  tests  and  operational  flight  testing.  However,  as 
discussed  in  dod’s  letter,  the  first  phase  will  be  conducted  primarily 
indoors  in  a  chamber  facility  at  Patuxent  River,  Maryland.  Thus,  most  of 
OT-IIA  will  not  be  done  in  a  realistic  operational  environment  as  required 
by  DOD  policy  and  should  therefore  not  be  considered  to  constitute 
realistic  operational  testing.  The  Navy’s  program  schedule  further 
indicates  that  the  second  phase  of  testing  (OT-IIB),  which  is  planned  for  a 
4-month  period  at  the  Navy’s  open  air  range  at  China  Lake,  California,  is  to 
take  place  after  low-rate  production  begins. 

2.  Requiring  the  Navy  to  comply  with  our  recommendation  would  better 
ensure  that  the  Navy  does  not  commit  to  production  of  a  deficient  system, 
as  it  did  with  earlier  versions  of  the  ALR-67.  dod’s  compliance  with  the 
congressional  reporting  requirements,  including  mandatory  certification 
from  the  Director  of  Operational  Test  and  Evaluation  that  the  ALR-67(V)3 
is  potentially  operationally  effective  and  potentially  operationally  suitable, 
may  provide  some  assurance  of  future  successful  performance.  However, 
DOD  can  only  be  certain  that  the  ALR-67(V)3’s  problems  have  been  solved 
by  satisfactorily  completing  both  phases  of  testing  (OT-IIA  and  OT-IIB) 
and  determining  that  it  is  both  effective  and  suitable. 

3.  The  majority  of  the  flight  testing,  lasting  4  months  according  to  the 
Navy’s  schedule,  is  to  be  conducted  after  the  low-rate  production  decision. 

4.  We  believe  that  the  intent  to  comply  with  congressional  reporting 
requirements  will  ensure  that  the  ALR-67(V)3  is  better  tested  prior  to 
beginning  production  than  its  predecessor  ALR-67  models.  However, 
assurance  of  operational  effectiveness  and  suitability  can  only  be  achieved 
through  successful  completion  of  both  test  phases  (OT-IIA  and  B).  Despite 
the  various  congressional  reporting  requirements,  the  Navy  may  still 
commit  to  the  ALR-67(V)3’s  low-rate  production  before  demonstrating  in  a 
realistic  operational  environment  that  the  system’s  performance  is 
satisfactory  by  completing  both  phases  of  testing.  Compliance  with  our 
recommendation,  in  addition  to  the  reporting  requirements,  could  prevent 
premature  corrunitment  to  production. 
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